The after recording (hereinafter sometimes referred to simply as AR) is a method of audio edit processing in which after the original data (only the video data, or the video data plus audio data such as voices) are recorded in a studio or location, new audio data (AR audio data) such as effect sounds are added or recorded or a part of the original audio data is replaced with the AR audio data in the editing process.
Several methods as described in (1) to (3) below, for example, are available for after recording the original data recorded in a digital recording medium. In this specification, however, the methods (1) and (3) will be explained.
(1) While reproducing the original data, the AR sound is input, and this AR audio data is multiplexed with the original data in real time at the time of reproducing the original data and recorded in the digital recording medium.
(2) While reproducing the original data, the input AR audio data are temporarily recorded in another recording medium. After that (i.e. not in real time), the AR audio data reproduced from the secondary recording medium are multiplexed and recorded with the original data in the digital recording medium.(3) The AR audio data are recorded in the digital recording medium without being multiplexed with the original data, and at the time of reproduction, the original data read from the digital recording medium and the AR audio data are mixed and output as a final sound.
FIG. 1 shows an example of recording data (bit stream) in a digital recording medium. A video bit stream V, a primary audio bit stream A1 and a secondary audio bit stream A2 are multiplexed into a single stream and recorded. The bit rate of the audio bit stream is lower than that of the video bit stream (for example, the former is about 6 Mbps or 8 Mbps, whereas the latter is about 256 Kbps. Thus the ratio between them is about 25:1 or 32:1). As a result, a major proportion of the recording area is occupied by V, while the recording area for A1, A2 is periodically located as spots.
In the case where MPEG2 is used as a coding standard for a digital recording medium, for example, the video bit stream V compression-coded by video according to MPEG2, the primary audio bit stream A1 compression-coded by video according to MPEG2 and the secondary audio bit stream A2 compression-coded by audio according to MPEG2 are recorded in the packet multiplexing scheme by the MPEG2 system.
FIG. 2 shows an example of the process for conducting the after recording by the method (1) for the original data recorded in the digital recording medium as shown in FIG. 1. First, the video bit stream V and the primary audio bit stream A1 constituting the original data are read from a digital recording medium. After being demodulated and error corrected by a RF & demodulation circuit 21 and an ECC circuit 22, the bit streams are applied through a buffer 23 for absorbing the difference between the read rate and the decode processing rate. Then the bit streams are applied to a decoder (including a demultiplexer, a video decoder and an audio decoder) 24, so that the original image and sound are reproduced from a display and a speaker, respectively, not shown, through a video output terminal P1 and an audio output terminal P2.
The user wanting to conduct the AR operation reproduces the image and the sound in this way while inputting the AR audio data (such as the audio data generated before a microphone not shown or the audio data selectively reproduced from an AR audio library tape) by way of an audio input terminal P3. In many cases, the AR audio data actually recorded are the input AR audio data mixed with the original audio (primary audio) data in a mixer 25.
In synchronism with the reproduction of the original image and sound, the AR audio data are encoded in an encoder 26, and applied through a buffer for absorbing the difference between the encode processing rate and the write rate. Then, the correction code is added and the signal is modulated in an ECC circuit 28 and a modulation & RF circuit 29, respectively. After that, this AR audio data is written in the recording area of the secondary audio bit stream A2.
The recording area of the secondary audio bit stream A2 may have the capacity just sufficient to record the AR audio data. The audio data need not be recorded and the recording area may thus be vacant before the after recording process. In the case where the video bit stream V, the primary audio bit stream A1 and the secondary audio bit stream A2 fail to form into a correctly multiplexed bit stream (in compliance with the system specification of MPEG2) when writing the AR audio data in the vacant recording area of the secondary audio bit stream A2, then remultiplexing is necessary at the time of after recording.
The after recording process is thus completed. When reproducing images and sounds as a final product from the digital recording medium for which the after recording is completed, an arbitrary one of the original sound (primary audio bit stream A1) and the AR sound (the secondary audio bit stream A2) can be selected and reproduced.
The foregoing description for the processing assumes that only the AR audio data is written during the write operation for convenience sake. Actually, however, in view of the fact that the ECC circuit processes a predetermined size of data block as a unit, the original data is required to be rewritten at the same time that the AR audio data is written. Specifically, assuming that the packet size of the audio bit stream is 2 Kbytes while the size of the data block of the ECC circuit is 32 Kbytes or 64 Kbytes, the error correction code is required to be added in the ECC circuit 28 as data in units of 32 or 64 Kbytes covering the AR audio data and the video bit stream V recorded in the area adjacent to the recording area of the secondary audio bit stream A2 where the audio data is to be written. In this way, when writing new data in a part of the area in the digital recording medium, the error correction code is required to be added to correct the data also in the area adjacent thereto. Thus, the original data is required to be rewritten.
Incidentally, in the after recording operation performed through these processes, approximately the following processing time is consumed in each circuit after reading the video bit stream V and the primary audio bit stream A1 before writing the AR audio data.
ECC circuits 22, 28: Each about several tens of milliseconds
Buffers 23, 27: Each about several hundred milliseconds to several tens of seconds (considerably varied with the function of the buffer used)
Decoder 24: About several tens of milliseconds
Encoder 26: About several tens of milliseconds
As described above, the average time required for processing is several hundred to about several seconds (or several tens of seconds in some cases). Thus the delay problem has been posed that the AR audio data is written in an area corresponding to the reproduction time delayed the aforementioned processing time behind the area where the original data to be simultaneously reproduced is recorded. Specifically, in the case of packet multiplexing, for example, the AR audio data are recorded as packets having a time stamp indicating the time delayed behind the time indicated by the time stamp in the packets of the original data to be reproduced at the same time.
In addition, this length of the processing time is usually somewhat varied with each AR operation. Therefore, the temporal relation between the recording area of the original data and the recording area of the AR audio data is also varied with each session of after recording. As a result, in the prior art, it has been difficult to reproduce the AR audio data in complete temporal agreement with the original images or the like when reproducing the images and sounds as a final product.
Accordingly, an object of the present invention is to provide a method and an apparatus for after recording in which the AR audio data can be easily reproduced in complete temporal agreement with the original images or the like even after the processes as shown in FIG. 2 employing the AR technique of (1) above.
Another object of the invention is to provide an AR method and apparatus and an reproduction method and apparatus in which the AR audio data can be reproduced in complete temporal agreement with the original images or the like even when employing the AR technique of (3) above.